A Framework for Cost-Aware Decentralized Services

Network services catering to large client populations are inevitably decentralized to benefit from the improved performance, availability, and scalability that decentralization offers. Decentralized network services might be deployed as independent servers in different administrative domains (e.g., domain name system), distributed globally with a central management (e.g., content distribution networks such as Akamai), or hosted on tightly-connected server clusters at a single location (e.g., search engines for the web). Irrespective of the deployment model, however, the overall performance of these services depends on the available amount of resources, such as the number of nodes, network bandwidth, memory or storage space, etc. The sheer scale of some of these services makes over-provisioning, a practice commonly resorted to in the industry, prohibitively expensive, and causes judicious resource allocation crucial for their performance. These network services often allocate resources in terms of logical units or objects. For example, a DNS record, an image or multi-media file, or a web page might be units of resource allocation. A fundamental resource allocation problem for these decentralized network services involves deciding which node hosts which object — a problem, that can get out of hand for a largescale network service with millions of objects and thousands of nodes making previously-known, centralized resource allocation techniques [2] impractical. The common approach to resource allocation in decentralized services is to use heuristics based on local information to drive resource allocation. The widely-prevalent practice of caching in content distribution networks and other lookup services such as DNS, where content is cached passively in response to a request, exemplifies this approach. However, heavy-tailed distributions of workload characteristics, such as popularity, size, or rate of change of content, make less-informed heuristics ineffective. Note that there is a continuous tradeoff between the amount of resources allocated and the performance provided by a service. While it is possible to continuously improve the performance of a service by adding more nodes, network bandwidth, or memory, the marginal benefit decreases sharply as higher and higher performance levels are targeted. For example, in a web cache, Zipf popularity distribution means that highly popular objects can be cached using limited resources while achieving additional improvements in cache hit rate requires caching more and more objects, quickly exhausting available resources. Several studies have shown that passive web caches have limited performance in practice [1, 8]. We have developed a novel approach for resource management in distributed systems that bases its decisions on global information yet does not require any centralized coordination. Our approach [3] is analysis-driven and poses the resource allocation problem as a constrained optimization problem, where the constraint expresses the performance requirements of the network service. Servers hosting the network service solve the optimization problem individually; but, take into account global workload characteristics to find solutions to the optimization problem. Constrained optimization provides a fine-grained “knob” to tune the performance of the network service. The constraints to the problem could be either a performance target or a resource limit depending on the application. For example, a content distribution network might pose its resource allocation problem in two ways: maximize cache hit rate while limiting memory consumption at each node to a set limit or achieve a target average cache hit rate with as little memory as possible. The resource allocated and the performance metric targeted can be different for different applications. For example, a web monitoring tool that continuously crawls the web and updates the indexes of search engines might target to minimize update detection time while keeping the total bandwidth consumption within a set limit. Our approach seeks to satisfy global, system-wide performance goals. A tempting yet naı̈ve approach to resource allocation is to apply constrained optimization solely using locally available workload characteristics. Unfortunately, this approach suffers from the same limitations as passive caching outlined earlier in the presence of heavy-tailed distributions; it results in several times higher resource requirements compared to a centralized approach that is based on a global knowledge of the workload. Surprisingly, we find that using a “concise” summary of global workload characteristics at each node enables a